This application is based on Japanese patent application No. 9-206856 filed on Jul. 31, 1997, the entire contents of which are incorporated herein by reference.
a) Field of the Invention
The present invention relates to techniques of processing image signals, and more particularly to image signal processing techniques with interpolation.
b) Description of the Related Art
FIG. 11A shows a fraction of image data picked up with a charge coupled device (CCD) camera. Image data is constituted of red (R), green (G), and blue (B) signals. A three-chip type sensor is made of three sensors for R, G, and B signals. In a single chip type sensor, R, G, and B signals are disposed on one sensor in a matrix such as shown in FIG. 11A. Various types of matrices are known. FIG. 11A shows an example a Bayer matrix. In the Bayer matrix, R and G signals are alternately disposed in one row (horizontal line), and in the next adjacent rows, G and B signals are alternately disposed. Therefore, a row of R and G signals and a row of G and B signals are alternately disposed.
The example shown in FIG. 11A shows 3xc3x973 image data. There are four R signal pixels at the four corners of the 3xc3x973 unit. R signal at the other pixel positions cannot be obtained so that it is necessary to obtain R signal through interpolation. In order to obtain R signal at the pixel position of G1 signal, it is interpolated through averaging of R signals at the adjacent right and left two pixels. R signal at the pixel of G4 signal is interpolated in the similar manner. R signals at the pixels of G2 and G3 signals are interpolated through averaging of R signals at the adjacent upper and lower pixels. R signal at the pixel of B signal is interpolated through averaging of R signals of the obliquely adjacent four pixels (at four corners of the 3xc3x973 unit). B signals are also interpolated in the similar manner to R signals.
Next, interpolation for G signal will be described. The center pixel in the unit is B signal which has no information on G signal. It is therefore necessary to interpolate G signal. Assuming that G signal at the center pixel is Gc signal, this Gc signal can be obtained through averaging by the equation (1).
Gc=(G1+G2+G3+G4)/4xe2x80x83xe2x80x83(1)
where G1, G2, G3, and G4 signals are G signals at four pixels adjacent to Gc signal.
FIG. 11B shows 3xc3x973 image data containing one horizontal edge. Pixels with hatching in FIG. 11B indicate that they have larger signals than other pixels. If Gc signal at the center pixel is obtained through averaging of four pixels, the horizontal edge of the image data becomes unsharp. In order to avoid this, if image data contains a horizontal edge, Gc signal is obtained by the following equation (2).
Gc=(G2+G3)/2xe2x80x83xe2x80x83(2)
A method of judging whether or not 3xc3x973 image data contains a horizontal edge will be described. If image data contains an edge and the following conditional formula (3) is satisfied, it is judged that the image data contains a horizontal edge.
|G1xe2x88x92G4| greater than |G2xe2x88x92G3|xe2x80x83xe2x80x83(3)
FIG. 11C shows 3xc3x973 image data containing one vertical edge. Also in this image data, if Gc signal at the center pixel is obtained through averaging of four pixels, the vertical edge of the image data becomes unsharp. In order to avoid this, if image data contains a vertical edge, Gc signal is obtained by the following equation (4).
Gc=(G1+G4)/2xe2x80x83xe2x80x83(4)
A method of judging whether or not 3xc3x973 image data contains a vertical edge will be described. If image data contains an edge and the following conditional formula (5) is satisfied, it is judged that the image data contains a vertical edge.
|G1xe2x88x92G4| less than |G2xe2x88x92G3|xe2x80x83xe2x80x83(5)
In the examples of FIGS. 11B and 11C, the image data contains one edge. In this case, by obtaining GC signal by the equation (2) or (4), a resolution of the edge can be retained.
FIG. 11D shows 3xc3x973 image data containing two horizontal edges which are a horizontal edge between the first and second rows LN1 and LN2 and a horizontal edge between the second and third rows LN2 and LN3. It is not possible to judge from the conditional formulas (3) and (5) whether the image data has either a horizontal edge or a vertical edge, because the conditional formulas (3) and (5) become |G1xe2x88x92G4|=|G2xe2x88x92G3|.
Even if none of the conditional formulas (3) and (5) are satisfied, it is not proper to obtain Gc signal from the equation (1). It is rather preferable in this case to judge that the image data contains a horizontal edge and to obtain Gc signal from the equation (2). If Gc signal is obtained from the equation (1), only Gc signal takes a value different from the other values on one line constituting the second row LN2.
If one edge is contained in image data, proper interpolation of G signal is possible. However, if image data contains two edges, proper interpolation of G signal is impossible.
If image data contains noises, discrimination between horizontal and vertical edges may lead to a false judgement. If interpolation is performed with a false judgement of an edge, a proper interpolation is impossible.
With improper interpolation, an edge (contour) becomes unsharp and the resolution of an image is lowered. A pseudo color (essentially absent) is developed lowering reproductivity of an image.
G signal contains a large amount of luminance components and greatly influences a resolution. If an interpolation precision of G signal is increased, the resolution of an image can be improved. From this reason, the abovedescribed interpolation depending upon an edge direction is performed in interpolating G signal.
In contrast to this, since R and B signals less influence the resolution, it has been considered that a simple interpolation method is sufficient. R and G signals have been interpolated therefore by simple averaging of adjacent pixels.
As the interpolation precision of G signal is improved, the relative interpolation precision of R and B signals lowers. Namely, G, R and B signals are unbalanced. There is a limit in improvement on the image quality if the interpolation precision of only G signal is improved. If the interpolation precision of not only G signal but also R and B signals is improved, the image quality can be expected to be improved further.
It is an object of the present invention to provide image signal processing techniques capable of properly interpolating image signals having various patterns.
It is another object of the present invention to provide image signal processing techniques capable of properly interpolating image signals containing noises.
It is still another object of the present invention to provide image signal processing techniques capable of improving an interpolation precision of chrominance signals constituting an image.
According to one aspect of the present invention, there is provided an image signal processing apparatus comprising: first edge judgement means for judging from four pixels adjacent to an object pixel whether the object pixel constitutes a single edge; first edge direction judging means for judging whether the single edge is horizontal or vertical, if the first edge judging means judges that the object pixel constitutes the single edge; and interpolating means for interpolating the object pixel in accordance with at least right and left two pixels adjacent to the object pixel in a horizontal direction, if the first edge direction judging means judges that the single edge is horizontal, and for interpolating the object pixel in accordance with at least upper and lower two pixels adjacent to the object pixel in a vertical direction, if the first edge direction judging means judges that the single edge is vertical.
After it is judged whether the object pixel constitutes a single edge, the direction of the edge is judged so that the edge direction can be judged reliably. For example, the edge direction can be judged after the object pixel constituting two edges and the object pixel without edge are first excluded from signal processing. After the edge direction is judged reliably, the object pixel is interpolated so that a lowered resolution or generation of pseudo color can be prevented.
According to another aspect of the present invention, there is provided an image signal processing apparatus for sequentially interpolating each of a plurality of pixels in an image as the object pixel, comprising: edge direction judging means for judging whether an edge constituted by the object pixel is horizontal or vertical; storage means for storing an edge direction of each of the plurality of object pixels judged by the edge judging means; edge direction re-judging means for re-judging that the object pixel has a different direction if a majority of the edge directions of pixels adjacent to the object pixel and stored in the storage means has an edge direction different from an edge direction of the object pixel; and interpolating means for interpolating the object pixel in accordance with at least right and left two pixels adjacent to the object pixel in a horizontal direction, if the edge direction judging means or the edge direction re-judging means judges that the edge is horizontal, and for interpolating the object pixel in accordance with at least upper and lower two pixels adjacent to the object pixel in a vertical direction, if the edge direction judging means or the edge direction rejudging means judges that the edge is vertical.
After the edge direction of the object pixel is judged, the edge direction of the object pixel is re-judged in accordance with edge directions of nearby pixels. Therefore, even if a pixel contains noise, the edge direction can be judged reliably.
According to a further aspect of the present invention, there is provided an image signal processing apparatus comprising: chrominance signal acquiring means for acquiring at least green signals of some pixels of a one-dimensional pixel array and red or blue signals of other pixels; green signal interpolating means for obtaining green signals of all pixels through interpolation between pixels of the pixel array; first converting means for obtaining as a red color difference signal a difference between red and green signals of a same pixel in the pixel array and obtaining as a blue color difference signal a difference between blue and green signals of the same pixel; color signal interpolating means for interpolating the red color difference signal and the blue color difference signal by using pixels of the pixel array, in accordance with the read color difference signal and the blue color difference signal converted by the first converting means; and second converting means for obtaining the red signal and the blue signal by adding the green signal of the same pixel to the red color difference signal and the blue color difference signal interpolated by the color difference signal interpolating means.
Red and blue signals are interpolated by using not only the red and blue signals but also the green signal. An interpolation precision can therefore be improved. The green signals may include not only those acquired by the chrominance signal but also those interpolated by the green signal interpolating means. If the green signal interpolating means interpolates the green signals with high precision, also the red and blue signals can be interpolated with high precision. With the high precision interpolation of signals, the image quality can be improved and a lowered resolution and generation of pseudo colors can be prevented. interpolating means. If the green signal interpolating means interpolates the green signals with high precision, also the red and blur signals can be interpolated with high precision. With the high precision interpolation of signals, the image quality can be improved and a lowered resolution and generation of pseudo colors can be prevented.